13 research outputs found
Recommended from our members
New topic detection in microblogs and topic model evaluation using topical alignment
textThis thesis deals with topic model evaluation and new topic detection in microblogs. Microblogs are short and thus may not carry any contextual clues. Hence it becomes challenging to apply traditional natural language processing algorithms on such data. Graphical models have been traditionally used for topic discovery and text clustering on sets of text-based documents. Their unsupervised nature allows topic models to be trained easily on datasets meant for specific domains. However the advantage of not requiring annotated data comes with a drawback with respect to evaluation difficulties. The problem aggravates when the data comprises microblogs which are unstructured and noisy.
We demonstrate the application of three types of such models to microblogs - the Latent Dirichlet Allocation, the Author-Topic and the Author-Recipient-Topic model. We extensively evaluate these models under different settings, and our results show that the Author-Recipient-Topic model extracts the most coherent topics. We also addressed the problem of topic modeling on short text by using clustering techniques. This technique helps in boosting the performance of our models.
Topical alignment is used for large scale assessment of topical relevance by comparing topics to manually generated domain specific concepts. In this thesis we use this idea to evaluate topic models by measuring misalignments between topics. Our study on comparing topic models reveals interesting traits about Twitter messages, users and their interactions and establishes that joint modeling on author-recipient pairs and on the content of tweet leads to qualitatively better topic discovery.
This thesis gives a new direction to the well known problem of topic discovery in microblogs. Trend prediction or topic discovery for microblogs is an extensive research area. We propose the idea of using topical alignment to detect new topics by comparing topics from the current week to those of the previous week. We measure correspondence between a set of topics from the current week and a set of topics from the previous week to quantify five types of misalignments: \textit{junk, fused, missing} and \textit{repeated}. Our analysis compares three types of topic models under different settings and demonstrates how our framework can detect new topics from topical misalignments. In particular so-called \textit{junk} topics are more likely to be new topics and the \textit{missing} topics are likely to have died or die out.
To get more insights into the nature of microblogs we apply topical alignment to hashtags. Comparing topics to hashtags enables us to make interesting inferences about Twitter messages and their content. Our study revealed that although a very small proportion of Twitter messages explicitly contain hashtags, the proportion of tweets that discuss topics related to hashtags is much higher.Computer Science
Recommended from our members
Explainable improved ensembling for natural language and vision
Ensemble methods are well-known in machine learning for improving prediction
accuracy. However, they do not adequately discriminate among underlying
component models. The measure of how good a model is can sometimes be estimated
from “why” it made a specific prediction. We propose a novel approach
called Stacking With Auxiliary Features (SWAF) that effectively leverages component
models by integrating such relevant information from context to improve
ensembling. Using auxiliary features, our algorithm learns to rely on systems that
not just agree on an output prediction but also the source or origin of that output.
We demonstrate our approach to challenging structured prediction problems
in Natural Language Processing and Vision including Information Extraction, Object
Detection, and Visual Question Answering. We also present a variant of SWAF
for combining systems that do not have training data in an unsupervised ensemble
with systems that do have training data. Our combined approach obtains a new
state-of-the-art, beating our prior performance on Information Extraction.
The state-of-the-art systems on many AI applications are ensembles of deeplearning
models. These models are hard to interpret and can sometimes make odd
mistakes. Explanations make AI systems more transparent and also justify their
predictions. We propose a scalable approach to generate visual explanations for
ensemble methods using the localization maps of the component systems. Crowdsourced
human evaluation on two new metrics indicates that our ensemble’s explanation
significantly qualitatively outperforms individual systems’ explanations.Computer Science